Method and apparatus for encoding, decoding a video signal using additional control of quantizaton error

ABSTRACT

Disclosed herein is a method of encoding a video signal, comprising receiving an original video signal; comparing the original video signal with a previously reconstructed signal; generating a correction signal to minimize a sum of a distortion component and a rate component; and entropy-encoding the correction signal that is transmitted to the decoder for video signal reconstruction, wherein the previously reconstructed signal has been inverse-transformed by additionally using a scaling diagonal matrix.

TECHNICAL FIELD

The present invention relates to a method and apparatus for encoding anddecoding a video signal and, more particularly, to a coding technologyusing an additional control of quantization error.

BACKGROUND ART

Compression coding means a set of signal processing technologies forsending digitalized information through a communication line or storingdigitalized information in a form suitable for a storage medium. Media,such as videos, images, and voice may be the subject of compressioncoding. In particular, a technique for performing compression coding onvideos is called video compression.

Many media compression technologies are based on two approach methods:predictive coding and transform coding. In particular, a hybrid codingtechnology includes spatially predicting samples using previouslydecoded context values and performing transform coding on predictederrors. Such a process is performed on a Gaussian signal so that it hasan optimized a Rate Distortion (RD) value.

However, common video signals need to be more efficiently coded becausethey have structures not suitable for Gaussian signals.

Meanwhile, an importance may be different in each of errors occurring inother parts of a block. Accordingly, there is a need for a methodcapable of controlling errors in both a space domain and a frequencydomain.

DISCLOSURE Technical Problem

An embodiment of the present invention is directed to more efficientlycoding a signal having an edge and directional structure.

An embodiment of the present invention is directed to non-casuallypredicting a video signal using a transform-coded signal together with apredicted signal.

An embodiment of the present invention is directed to coding a videosignal based on non-orthogonal transform.

An embodiment of the present invention is directed to obtaining anoptimized transform coefficient that minimizes distortion.

An embodiment of the present invention is directed to deriving a RateDistortion (RD)-optimized quantization step size.

An embodiment of the present invention is directed to representing anon-casual coding technology to which the present invention may beapplied using non-orthogonal transform having a form and parameters.

An embodiment of the present invention is directed to controlling aquantization error in both a space domain and a frequency domain.

An embodiment of the present invention is directed to defining differentdiagonal matrices in order to differentiate the importance of errors onthe space domain.

An embodiment of the present invention is directed to proposing a methodof calculating optimized diagonal matrices from a viewpoint of RateDistortion (RD).

An embodiment of the present invention is directed to proposing a methodof more finely controlling a quantization error on the space domain.

Technical Solution

The present invention proposes a method of more efficiently coding asignal having an edge and directional structure.

Furthermore, the present invention proposes a method of non-casuallypredicting a video signal using a transform-coded signal together with apredicted signal.

Furthermore, the present invention proposes a method of coding a videosignal based on non-orthogonal transform.

Furthermore, the present invention proposes a quantization algorithm forobtaining an optimized transform coefficient.

Furthermore, the present invention proposes a method of deriving anoptimized quantization step size.

Furthermore, the present invention proposes a non-casual codingtechnology that may be represented by non-orthogonal transform having aform and parameters.

Furthermore, the present invention proposes a method of generating anoptimized prediction signal using all the already reconstructed signalsand a context signal.

Furthermore, the present invention proposes a method of controlling aquantization error in both a space domain and a frequency domain.

Furthermore, the present invention defines different diagonal matricesin order to differentiate the importance of errors on the space domain.

Furthermore, the present invention proposes a method of calculatingoptimized diagonal matrices from a viewpoint of Rate Distortion (RD).

Furthermore, the present invention proposes a method of more finelycontrolling a quantization error on the space domain.

Advantageous Effects

The present invention can perform more elaborate and advanced predictionusing all the decoded information.

Furthermore, the present invention can code a signal having an edge anddirectional structure more efficiently by non-casually predicting avideo signal using a transform-coded signal together with a predictedsignal.

Furthermore, the present invention can perform more elaborate andadvanced prediction by proposing a non-casual coding technology that maybe represented by non-orthogonal transform having a form and parameters.

Furthermore, the present invention can minimize a quantization error byproposing a quantization algorithm for obtaining an optimized transformcoefficient.

Furthermore, the present invention can perform more advanced coding byproposing a method of deriving an optimized quantization step size.

Furthermore, the present invention can generate an optimized predictionsignal using all the already reconstructed signals and a context signal.

Furthermore, the present invention can perform more advanced coding bycontrolling a quantization error in both a space domain and a frequencydomain.

DESCRIPTION OF DRAWINGS

FIGS. 1 and 2 are schematic block diagrams of an encoder and a decoderin which video coding is performed;

FIGS. 3 and 4 illustrate embodiments to which the present invention maybe applied and are schematic block diagrams of an encoder and a decoderto which an advanced coding method has been applied;

FIGS. 5 and 6 illustrate embodiments to which the present invention maybe applied and define layers illustrating a method of performingprediction using previously coded pixels;

FIG. 7 illustrates an embodiment to which the present invention may beapplied and is a flowchart illustrating a method of performingprediction using previously coded pixels for each layer;

FIG. 8 illustrates an embodiment to which the present invention may beapplied and is a flowchart illustrating a quantization process forobtaining an optimized coefficient;

FIG. 9 illustrates an embodiment to which the present invention may beapplied and is a detailed flowchart illustrating a quantization processfor obtaining an optimized coefficient;

FIG. 10 illustrates an embodiment to which the present invention may beapplied and is a flowchart illustrating a process of obtaining anoptimized quantization step size;

FIGS. 11 and 12 illustrate embodiments to which the present inventionmay be applied, wherein FIG. 11 illustrates test images to which thepresent invention has been applied and FIG. 12 illustrates percentagesof rate gains to test images;

FIG. 13 illustrates an embodiment to which the present invention may beapplied and is a schematic flowchart illustrating an improved predictivecoding method;

FIG. 14 illustrates an embodiment to which the present invention may beapplied and is a schematic flowchart illustrating a method of performingquantization based on an optimized quantization step size;

FIGS. 15 and 16 illustrate embodiments to which the present inventionmay be applied and are schematic block diagrams of an encoder and adecoder to which an advanced coding method has been applied throughcontrol of a quantization error;

FIG. 17 illustrates an embodiment to which the present invention may beapplied and is a flowchart illustrating a process of obtaining a scalingdiagonal matrix through a Rate Distortion (RD) optimization process;

FIG. 18 illustrates an embodiment to which the present invention may beapplied and is a graph illustrating a comparison between the codinggains of respective images in the case in which coding is performedusing an optimized scaling matrix and the case in which coding isperformed using an existing method;

FIGS. 19 and 20 illustrate embodiments to which the present inventionmay be applied and are schematic block diagrams of an encoder and adecoder to which an advanced coding method has been applied; and

FIG. 21 illustrates an embodiment to which the present invention may beapplied and is a schematic flowchart illustrating an advanced videocoding method.

BEST MODE

An embodiment of the present invention provides a method of encoding avideo signal, comprising receiving an original video signal; comparingthe original video signal with a previously reconstructed signal;generating a correction signal to minimize a sum of a distortioncomponent and a rate component; and entropy-encoding the correctionsignal that is transmitted to the decoder for video signalreconstruction, wherein the previously reconstructed signal has beeninverse-transformed by additionally using a scaling diagonal matrix.

In an aspect of the present invention, the correction signal isgenerated based on another diagonal matrix being used to differentiate aweighting of errors in a spatial domain.

In an aspect of the present invention, the method further includescalculating an optimal set of multiple diagonal matrices including thescaling diagonal matrix, wherein the correction signal is generatedbased on the optimal set of multiple diagonal matrices.

In an aspect of the present invention, the optimal set of multiplediagonal matrices is encoded as side information, and is transmitted toa decoder.

In an aspect of the present invention, the optimal set of multiplediagonal matrices is encoded before encoding frames of the originalvideo signal.

In an aspect of the present invention, the distortion component isindicative of total distortion between the original video signal and areconstructed signal, and the rate component is indicative of a numberof bits required to send a quantized coefficient.

Another embodiment of the present invention provides a method ofdecoding a video signal, comprising: receiving the video signalincluding a correction signal; reading side information includingmultiple diagonal matrices from the video signal; obtaining thecorrection signal by entropy-decoding the video signal; andreconstructing a signal based on the correction signal and the multiplediagonal matrices.

In an aspect of the present invention, the multiple diagonal matricesincludes a scaling diagonal matrix.

In an aspect of the present invention, the method further includesperforming an inverse-transform to the correction signal by additionallyusing the scaling diagonal matrix.

In an aspect of the present invention, the correction signal includes anoptimal coefficient value which minimizes a sum of a distortioncomponent and a rate component.

In an aspect of the present invention, the multiple diagonal matrices isread before decoding frames of the video signal.

Another embodiment of the present invention provides an apparatus ofencoding a video signal, comprising: a receiving unit configured toreceive an original video signal; an optimizer configured to compare theoriginal video signal with a previously reconstructed signal, andgenerate a correction signal to minimize a sum of a distortion componentand a rate component; and an entropy-encoding unit configured toentropy-encode the correction signal that is transmitted to a decoderfor video signal reconstruction, wherein the previously reconstructedsignal has been inverse-transformed by additionally using a scalingdiagonal matrix.

In an aspect of the present invention, wherein the optimizer furtherconfigures to calculate an optimal set of multiple diagonal matricesincluding the scaling diagonal matrix, wherein the correction signal isgenerated based on the optimal set of multiple diagonal matrices.

Another embodiment of the present invention provides an apparatus ofdecoding a video signal, comprising: a receiving unit configured toreceive the video signal including a correction signal, and read sideinformation including multiple diagonal matrices from the video signal;an entropy-decoding unit configured to obtain the correction signal byentropy-decoding the video signal; and a reconstruction unit configuredto reconstruct a signal based on the correction signal and the multiplediagonal matrices.

In an aspect of the present invention, the decoding apparatus furthercomprises an inverse-transform unit configured to perform aninverse-transform to the correction signal by additionally using ascaling diagonal matrix.

MODE FOR INVENTION

Hereinafter, exemplary elements and operations in accordance withembodiments of the present invention are described with reference to theaccompanying drawings. It is however to be noted that the elements andoperations of the present invention described with reference to thedrawings are provided as only embodiments and the technical spirit andkernel configuration and operation of the present invention are notlimited thereto.

Furthermore, terms used in this specification are common terms that arenow widely used, but in special cases, terms randomly selected by theapplicant are used. In such a case, the meaning of a corresponding termis clearly described in the detailed description of a correspondingpart. Accordingly, it is to be noted that the present invention shouldnot be construed as being based on only the name of a term used in acorresponding description of this specification and that the presentinvention should be construed by checking even the meaning of acorresponding term.

Furthermore, terms used in this specification are common terms selectedto describe the invention, but may be replaced with other terms for moreappropriate analysis if such terms having similar meanings are present.For example, a signal, data, a sample, a picture, a frame, and a blockmay be properly replaced and interpreted in each coding process.

FIGS. 1 and 2 illustrate schematic block diagrams of an encoder and adecoder in which media coding is performed.

The encoder 100 of FIG. 1 includes a transform unit 110, a quantizationunit 120, a dequantization unit 130, an inverse transform unit 140, abuffer 150, a prediction unit 160, and an entropy encoding unit 170. Thedecoder 200 of FIG. 2 includes an entropy decoding unit 210, adequantization unit 220, an inverse transform unit 230, a buffer 240,and a prediction unit 250.

The encoder 100 receives the original video signal and generates aprediction error by subtracting a predicted signal, output by theprediction unit 160, from the original video signal. The generatedprediction error is transmitted to the transform unit 110. The transformunit 110 generates a transform coefficient by applying a transformscheme to the prediction error.

The transform scheme may include, a block-based transform method and animage-based transform method, for example. The block-based transformmethod may include, for example, Discrete Cosine Transform (DCT) andKarhuhen-Loeve Transform. The DCT means that a signal on a space domainis decomposed into two-dimensional frequency components. A patternhaving lower frequency components toward an upper left corner within ablock and higher frequency components toward a lower right corner withinthe block is formed. For example, only one of 64 two-dimensionalfrequency components that is placed at the top left corner may be aDirect Current (DC) component and may have a frequency of 0. Theremaining frequency components may be Alternate Current (AC) componentsand may include 63 frequency components from the lowest frequencycomponent to higher frequency components. To perform the DCT includescalculating the size of each of base components (e.g., 64 basic patterncomponents) included in a block of the original video signal, the sizeof the base component is a discrete cosine transform coefficient.

Furthermore, the DCT is transform used for a simple expression into theoriginal video signal components. The original video signal is fullyreconstructed from frequency components upon inverse transform. That is,only a method of representing video is changed, and all the pieces ofinformation included in the original video in addition to redundantinformation are preserved. If DCT is performed on the original videosignal, DCT coefficients are crowded at a value close to 0 unlike in theamplitude distribution of the original video signal. Accordingly, a highcompression effect can be obtained using the DCT coefficients.

The quantization unit 120 quantizes the generated transform coefficientand sends the quantized coefficient to the entropy encoding unit 170.The entropy encoding unit 170 performs entropy coding on the quantizedsignal and outputs an entropy-coded signal.

The quantization unit 120 maps a specific range of input values forinput data to a single representative value. Quantization may becomputed by dividing the input data by a quantization step size as inthe following equation 1.

$\begin{matrix}{Y = {{{Sign}(X)}*{Round}\left\{ \frac{X}{Q} \right\}}} & \left\lbrack {{Equation}\mspace{14mu} 1} \right\rbrack\end{matrix}$

In Equation 1, Y denotes quantized data, X denotes input data, and Qdenotes a quantization step size. A function Sign( ) is operation forobtaining the sign of data, and a function Round( ) denotes round-offoperation. The quantization step size may be represented by aquantization range. Furthermore, in this specification, the quantizationstep size may mean a scaling parameter. When video coding is performed,a quantization step size may be changed. A compression ration may becontrolled using the changed quantization step size. Meanwhile, aquantization parameter using an integer value may be use instead of thequantization step size.

In a quantization process, as in the following equation 2, a quantizedcoefficient C′ may be obtained by dividing an input transformcoefficient C by a quantization step size Q.

C′=C/Q  [Equation 2]

In Equation 2, C′ denotes a quantized coefficient, C denotes an inputtransform coefficient, and Q denotes a quantization step size.

Meanwhile, the quantized signal output by the quantization unit 120 maybe used to generate a prediction signal. For example, the dequantizationunit 130 and the inverse transform unit 140 within the loop of theencoder 100 may perform dequantization and inverse transform on thequantized signal so that the quantized signal is reconstructed into aprediction error. A reconstructed signal may be generated by adding thereconstructed prediction error to a prediction signal output by theprediction unit 160.

The buffer 150 stores the reconstructed signal for the future referenceof the prediction unit 160. The prediction unit 160 generates aprediction signal using a previously reconstructed signal stored in thebuffer 150.

The decoder 200 of FIG. 2 receives a signal output by the encoder 100 ofFIG. 1. The entropy decoding unit 210 performs entropy decoding on thereceived signal. The dequantization unit 220 obtains a transformcoefficient from the entropy-decoded signal based on information about aquantization step size. The inverse transform unit 230 obtains aprediction error by performing inverse transform on the transformcoefficient. A reconstructed signal is generated by adding the obtainedprediction error to a prediction signal output by the prediction unit250.

The dequantization unit 220 may compute reconstructed data bymultiplying quantized data by a dequantization scaling value Q as in thefollowing equation 3.

X′=Y*Q  [Equation 3]

In Equation 3, X′ denotes reconstructed data, Y denotes quantized data,and Q denotes a dequantization scaling value. The dequantization scalingvalue Q may have the same value as a quantization step size.

The buffer 240 stores the reconstructed signal for the future referenceof the prediction unit 250. The prediction unit 250 generates aprediction signal using a previously reconstructed signal stored in thebuffer 240.

The present invention provides an intra prediction method in a hybridvideo coder. Sample values to be compressed are predicted usingpreviously coded context values, and predicted errors aretransform-coded. Such a process may be performed on a Gaussian signal sothat it has an optimized RD value. Common video signals include manysignals not suitable for Gaussian signals. Accordingly, the presentinvention is targeted to such signals and proposes a technology fornon-casually predicting each sample using a transform-coded sample and acontext value together with a prediction sample. Such non-causalencoding may be represented by non-orthogonal transform including a formand parameters.

FIGS. 3 and 4 illustrate embodiments to which the present invention maybe applied and are schematic block diagrams of an encoder and a decoderto which an advanced coding method has been applied.

The encoder 300 of FIG. 3 includes an optimizer (310), a quantizationunit 315, an inverse transform unit 320, a prediction unit 330, areconstruction unit 340, a buffer 350, and an entropy encoding unit 360.The decoder 400 of FIG. 4 includes an entropy decoding unit 410, adequantization unit 420, an inverse transform unit 430, a reconstructionunit 440, a buffer 450, and a prediction unit 460.

The optimizer 310 may fetch at least one of information about the pixelsof a current block, information about the pixels of a previously decodedblock, and information about a quantization step size from the buffer350. In this case, the pixel information of the current block may beindicative of the pixels of a block to be coded that are arranged into avector. The pixel information of a previously decoded block may beindicative of the pixels of a previously decoded block that are arrangedinto a vector. The quantization step size information may be indicativeof a quantization step size arranged into a vector.

The optimizer 310 may obtain a transform coefficient C(i,j) based on atleast one of the pixel information of the current block, the pixelinformation of the previously decoded block, and the quantization stepsize information. The transform coefficient C(i,j) may mean adequantized transform coefficient.

The inverse transform unit 320 may receive the obtained transformcoefficient C(i,j) and perform inverse transform on the receivedtransform coefficient C(i,j). The inverse transform unit 320 may obtaina residual signal “res(i,j)” by performing inverse transform on thereceived transform coefficient C(i,j).

The prediction unit 330 may fetch information about the pixels of thepreviously decoded block from the buffer 350. The prediction unit 330may predict the pixels of a current layer using at least one of thepixels of the previously decoded block and pixels reconstructed from aprevious layer. The prediction unit 330 may obtain a prediction signal“pred(i,j)” by performing such prediction. In this case, assuming that acurrent block is a B×B block, a horizontal component is j, and avertical component is i, the pixels of the current layer L_(k) may beindicative of pixels placed at positions (k,i) and (j,k) (i=1, . . . ,B, j=1, . . . , B, k=1, . . . , B). In this case, the pixelsreconstructed from the previous layer may be indicative of thereconstructed pixels of all the previous layers L₁, . . . , L_(k-1).This is described in more detail with reference to FIGS. 5 and 6.

The reconstruction unit 340 may obtain a reconstructed signal “rec(i,j)”by adding the prediction signal “pred(i,j)” obtained by the predictionunit 330 and the residual signal “res(i,j)” obtained by the inversetransform unit 320. In this case, the reconstructed signal “rec(i,j)”may mean the reconstructed signal of the current layer L_(k). Thereconstructed signal “rec(i,j)” is transmitted to the buffer 350 for thefuture prediction of a next layer.

The transform coefficient C(i,j) obtained by the optimizer 310 istransmitted to a quantization unit 315.

The quantization unit 315 performs a quantization process and transmitsquantized transform coefficient to the entropy encoding unit 360.

In this case, the transform coefficient C(i,j) may mean a RateDistortion (RD)-optimized transform coefficient. Furthermore, thequantization process may be performed by dividing the transformcoefficient C(i,j) by the quantization step size.

The entropy encoding unit 360 may receive the quantized transformcoefficient and perform entropy encoding on the received transformcoefficient.

The decoder 400 of FIG. 4 may receive a signal output by the encoder 300of FIG. 3.

The entropy decoding unit 410 may receive a bit stream and performentropy decoding on the bit stream.

The dequantization unit 420 may obtain a transform coefficient from theentropy-decoded signal using quantization step size information.

The inverse transform unit 430 may obtain the residual signal “res(i,j)”by performing inverse transform on the transform coefficient.

The reconstruction unit 440 may obtain the reconstructed signal“rec(i,j)” by adding the residual signal “res(i,j)” and the predictionsignal “pred(i,j)” obtained by the prediction unit 450. Thereconstructed signal “rec(i,j)” may be transmitted to the buffer 450 andstored therein. Furthermore, the reconstructed signal “rec(i,j)” may betransmitted to the prediction unit 450 for the future prediction of anext signal.

The embodiments described with reference to the encoder 300 of FIG. 3may be applied to the operations of the elements of the decoder 400 ofFIG. 4.

A hybrid video coder to which the present invention may be appliedperforms efficient predictive coding by spatially predicting samplesusing previously decoded samples (i.e., context values) and performingtransform coding on predicted errors.

In such a coding method, block transform is consecutively performed oneven signals whose block transform has been partially optimized. Forexample, the partially optimized signals may include signals havingsignificant inter-block correlations and signals having edge anddifferent directional singularities. Accordingly, a spatial predictionoperation may be considered to be less adaptive to an elaborateprediction process because it generates a prediction signal moreadaptive to simple transform compression. Efficiency of such aprediction operation may be strongly dependent on basic processes havinga Gaussian signal because the prediction operation is performed usingcontext values.

For detailed discussions, a one-dimensional example in which a sequencex_(i) (i=1˜N) is compressed using a context sample x₀ is taken intoconsideration.

For example, x includes a series of horizontal or directional pixelsfrom a target block on which directional prediction is to be performedusing a context sample x₀. In this case, the context sample x₀ may beobtained from the boundary of a previously decoded block. The contextsample x₀ is assumed to be available in both an encoder and a decoder.Assuming that linear prediction of x_(i) using the context sample x₀ isP_(i)(x₀), a residual signal “r_(i)” may be defined as in the followingequation 4.

r _(i) =x _(i) −P _(i)(x ₀)  [Equation 4]

The residual signal “r_(i)” may be represented as in the followingequation 5 after it is subjected to transform coding according to acoding process, subjected to transform decoding according to a decodingprocess.

{circumflex over (x)} _(i) =P _(i)(x ₀)+{circumflex over (r)}_(i)  [Equation 5]

In Equation 5, {circumflex over (x)}_(i) denotes a reconstructed signal.

If an optimized linear predictor is obtained and KLT is used, theprocess may become gradually optimal for the compression of a Gaussiansequence. However, the process may not be suitable for many image/videostructures, such as Gaussian modeling. Accordingly, in an embodiment ofthe present invention, a prediction method may be further improved usinga better predictor using all the decoded information during a decodingprocess. The present invention may have an excellent effect for a videosignal having an edge and directional structure.

Hereinafter, first, a basic idea is described through a one-dimensionalexample of the present invention. Next, connections between the presentinvention and DPCM are discussed by focusing on a linear predictor, andequivalent non-orthogonal transform is to be derived. Furthermore, aftera codec design is discussed, compression using non-orthogonal transformand the derivation of a Rate Distortion (RD)-optimized quantizationparameter are described. Finally, the details of simulation results towhich the present invention may be applied are described.

After transform decoding, the decoder may have access to all of theresidual samples. However, it only uses x₀ and r_(i) when decoding thei^(th) sample, {circumflex over (x)}_(i). In particular, when decoding{circumflex over (x)}_(i+1), the decoder has already reconstructed{circumflex over (x)}₁, which is typically a far better predictor of{circumflex over (x)}_(i+1) compared to x0. In the present invention,the decoding chain may be designed as following equation 6.

transform-decode=>{circumflex over (r)}=>{circumflex over (x)} _(i) =P^(t)/(x ₀ ,{circumflex over (r)} ₁ , . . . ,{circumflex over (r)}_(N))+{circumflex over (r)} _(i).  [Equation 6]

Since the decoder has all of the transform-decoded residuals available,this chain and the augmented predictor P^(t) may be feasible. Thecorresponding encoding chain can be described as the selection ofoptimal coded transform coefficients which, when fed into the transformdecoder in equation 6, result in {circumflex over (x)} that has theminimum distortion at a given target bit-rate.

While the present invention can be generalized to nonlinear predictionfunctions, it will keep the computationally simple, linear predictorsbut accomplish prediction using the closest available samples ratherthan using x0 everywhere. For the one-dimensional example, the presentinvention can construct equation 7.

$\begin{matrix}{{{{\hat{x}}_{1} = {{{P_{1}\left( x_{0} \right)} + {\hat{r}}_{1}} = {x_{0} + {\hat{r}}_{1}}}},{{\hat{x}}_{2} = {{P_{2}\left( {\hat{x}}_{1} \right)} = {{\hat{r}}_{2} = {x_{0} + {\hat{r}}_{1} + {\hat{r}}_{2}}}}},\vdots}{{\hat{x}}_{N} = {{{P_{N}\left( {\hat{x}}_{N - 1} \right)} + {\hat{r}}_{N}} = {x_{0} + {\hat{r}}_{1} + \ldots + {\hat{r}}_{N}}}}} & \left\{ {{Equation}\mspace{14mu} 7} \right\rbrack\end{matrix}$

In this case, the prediction may be linear with a prediction weight ofunity. In this setting, the prediction P_(i)(x₀) in equation 7 may besimply replaced with P_(i)({circumflex over (x)}_(i−1)). Other weightsand types of linear predictors may be straightforward generalizations.

Hereinafter, it will be explained about Relationship to DPCM andEquivalent Non-orthogonal Transforms

The equation 7 resembles a first-order DPCM decoder that is operatingwith a prediction weight of unity. While a DPCM system will encode theresiduals causally and independently, the decoder of equation 7corresponds to decoding of residuals that have been encoded non-causallyand jointly. This is due to {circumflex over (r)} being the output ofthe transform decoder shown in equation 6. It can be said that theproposed system gains the prediction accuracy of a DPCM system whileexploiting residual dependencies and other DPCM R-D inefficiencies viatransform coding.

Equation 7 can lead to the matrix equation 8.

{circumflex over (x)}=F{circumflex over (r)}+Bx ₀  [Equation 8]

Here, F is a (N×N) lower triangular prediction matrix with equation 9.

$\begin{matrix}{F_{i,j} = \left\{ \begin{matrix}{1,} & \left( {i \geq j} \right) \\{0,} & {otherwise}\end{matrix} \right.} & \left\lbrack {{Equation}\mspace{14mu} 9} \right\rbrack\end{matrix}$

This embodiment is a (N×1) matrix with unit entries. Augmenting equation8 to accommodate transform coding, the present invention can result inequation 10.

{circumflex over (x)}=FTĉ+Bx ₀,  [Equation 10]

In equation 10, T (N×N) is the transform used in compression (e.g., theblock DCT/DST in HEVC) and ĉ are the dequantized transform coefficients.Letting G=FT, equation 10 corresponds to the transform coding of{circumflex over (x)}−Bx₀ with the non-orthogonal transform G viaequation 11.

{circumflex over (x)}−Bx ₀ =Gĉ,  [Equation 11]

In this simple linear form, the present invention may be the transformcompression of x−Bx₀ using the non-orthogonal transform G.

Using mode-based linear predictors, the proposed decoding chain can beincorporated within a baseline hybrid codec like HEVC by designing F andB matrices and deriving the equivalent non-orthogonal transform G foreach prediction mode.

Such a decoding chain will have only a marginal complexity increasecompared to the baseline since all it will do is predict using theclosest samples rather than the boundary samples. The encoding chain ismore complex, however, because it must pick optimal coefficients totransmit for the decoding chain. Hereinafter, the present invention willprovide an iterative quantization algorithm which the encoder must carryout and derive rate-distortion optimal quantization parameters.

FIGS. 5 and 6 illustrate embodiments to which the present invention maybe applied and define layers illustrating a method of performingprediction using previously coded pixels.

An embodiment of the present invention provides a method of non-casuallypredicting a sample using previously coded pixels.

In this case, the pixels of a current block and previously coded pixelsused for prediction may be determined using various methods.

In an embodiment to which the present invention may be applied, acurrent block may be decomposed in at least one layer unit. Accordingly,the previously coded pixels may be determined in each layer unit.

In this case, the layer unit may be variously defined based on placedpixels according to a specific criterion. For example, pixels arrangedin horizontal and vertical directions based on pixels placed at the lefttop of a current block may be defined as a single layer. Furthermore,pixels arranged in the diagonal direction of a pixel placed at the lefttop may be defined as consecutive layers.

In this case, the layer may be defined as one pixel or a plurality ofpixels or may be defined as all the pixels of a block. Furthermore, thelayer may be defined as a set of consecutive pixels as illustrated inFIG. 5, but may be defined as a set of pixels that are not consecutiveaccording to circumstances.

For example, referring to FIG. 5, it is assumed that a current block isa B×B block and the position of a pixel within the block is (i,j). Inthis case, i∈{1, 2, . . . , B}, j∈{1, 2, . . . , B}. In this case,pixels arranged in horizontal and vertical directions based on a pixelplaced at the left top of the current block may be defined as a layerL₁. That is, a pixel placed at pixel positions (1,j) and (i,1) may bedefined as the layer L₁.

This may be generalized as follows. For example, a pixel placed at pixellocations (k,j) and (i,k) may be defined as a layer L_(k) (k=1, 2, . . ., B).

In an embodiment to which the present invention may be applied,previously coded pixels may include the pixels of a layer that is codedright before a layer to be coded.

Referring to FIG. 6, in order to predict a current layer L_(k), a layerL_(k-1) coded right before the current layer L_(k) may be used. In thiscase, in order to predict the current layer L_(k), pixels neighboringthe boundary of the current block may also be used. That is, pixels thatneighbor an already decoded block neighboring the current block may beused to predict the layer L_(k).

For example, the current layer L_(k) may be predicted based on thereconstructed pixels of all the previous layers L₁, . . . , L_(k-1) andpixels that neighboring an already decoded block.

Another embodiments of the present invention can provide predictionformation.

The encoder can arrange coeffs(i,j), i∈{1, 2, . . . , B}, j∈{1, 2, . . ., B} into a vector c. It can be represented as equation 12.

C((i−1)*B+(j−1)+1)=coeffs(i,j)  [Equation 12]

And, the encoder can arrange res(i,j), i∈{1, 2, . . . , B}, j∈{1, 2, . .. , B} into a vector r. It can be represented as equation 13.

r((i−1)*B+(j−1)+1)=res(i,j)  [Equation 13]

And then, the encoder can arrange pixels from previously decoded blocksinto a vector y.

In this case, the present invention can be implemented using matrixmultiplication as equation 14.

{tilde over (X)}=Fr+Hy, where {tilde over (X)} is the reconstructedblock  [Equation 14]

Also, the present invention can be implemented using matrixmultiplication as equation 15.

{tilde over (X)}=FTc+Hy, where T is the matrix equivalent of the inversetransform  [Equation 15]

Furthermore, the present invention can be implemented using matrixmultiplication as equation 16.

{tilde over (X)}=Gc+Hy, where G=FT  [Equation 16]

Furthermore, the present invention can be implemented using matrixmultiplication as equation 17.

{tilde over (X)}=G′c+H′y, where G′=F′T, and F′ and H′ are matricesoptimized over training sets  [Equation 17]

Meanwhile, the embodiments described with reference to FIGS. 5 and 6 maybe applied to intra prediction and may also be applied to variousprediction modes for intra prediction. However, the present invention isnot limited thereto. For example, the embodiments may also be applied tointer prediction.

FIG. 7 illustrates an embodiment to which the present invention may beapplied and is a flowchart illustrating a method of performingprediction using previously coded pixels for each layer.

First, an entropy-coded coefficient may be extracted from a received bitstream. Entropy decoding may be performed on the entropy-codedcoefficient at step S710 and the entropy-decoded coefficient may bedequantized at step S720, thereby being capable of obtaining a transformcoefficient “coeffs(i,j)”.

A residual signal “res(i,j)” may be obtained by performing inversetransform on the transform coefficient at step S730. The residual signal“res(i,j)” is used to reconstruct a current layer L_(k).

In order to predict the pixels of the current layer L_(k), the pixels ofa previously decoded block may be used. In this case, the pixels of thecurrent layer L_(k) may be predicted using the reconstructed pixels ofall the previous layers L₁, . . . , L_(k-1) together at step S740.

A prediction signal “pred(i,j)” generated at step S740 may be added tothe residual signal “res(i,j)” obtained at step S730, thereby beingcapable of reconstructing the pixels of the current layer L_(k) at stepS750. A reconstructed signal “rec(i,j)” generated as described above maybe used to predict a next layer.

FIG. 8 illustrates an embodiment to which the present invention may beapplied and is a flowchart illustrating a quantization process forobtaining an optimized coefficient. The present invention provides acompression method with non-orthogonal transforms.

Consider the random vector x (N×1). For notational convenience assumethat the context prediction is absorbed within x. The vector x isrepresented using the linear transform G (N×N), whose columns g_(i),i=1, . . . , N form the transform basis. Assume G is full rank but isotherwise general, i.e., G may be not necessarily orthogonal and g_(i)may be not necessarily unit norm.

x=Gc  [Equation 18]

In equation 18, c (N×1) are the transform coefficients. The coefficientsmay be scalar quantized to yield ĉ=Q(c) which are then entropy coded andtransmitted to a decoder.

The scalar quantization problem with respect to the non-orthogonal basisG where one aims to minimize the quantization distortion can be writtenas equation 19.

∥x−Gĉ∥  [Equation 19]

While the present invention can accommodate a variety of quantizers forcompatibility with video coders, it will be assumed as equation 20.

c=Λι  [Equation 20]

In equation 20 ι (N×1) is a vector of integers and Λ is a diagonalmatrix of quantizer step-sizes, i.e., Λ_(i,j)=λ_(i)δ_(i,j) with Δ_(i)the i^(th) step-size and δ_(i,j) is the Kronecker delta function.Equation 21 hence may be derived.

∥x−GΛι∥  [Equation 21]

Equation 21 can be recognized as a lattice quantizer whose optimalsolution in terms of L requires solving an integer problem. Manysuboptimal techniques have been proposed for the solution of equation19. In order to accommodate fast solutions, the present invention canincorporate a method similar to where one iteratively solves scalarquantization problems concentrating on each coefficient in turn. Assumeall coefficients except for the i^(th) coefficient have been quantized.The error vector can be defined as equation 22.

$\begin{matrix}{ɛ_{i} = {x - {\sum\limits_{\{{{k|{1 \leq k \leq N}},{k \neq i}}\}}{g_{k}{{\hat{c}}_{k}.}}}}} & \left\lbrack {{Equation}\mspace{14mu} 22} \right\rbrack\end{matrix}$

Without the integer constraint, the distortion may be minimized bychoosing the i^(th) coefficient to be equation 23.

$\begin{matrix}{c_{i}^{*} = {{\arg {\min\limits_{d}{{ɛ_{i} = {g_{i}d}}}^{2}}} = {g_{i}^{T}{ɛ_{i}/\left( {g_{i}^{T}g_{i}} \right)}}}} & \left\lbrack {{Equation}\mspace{14mu} 23} \right\rbrack\end{matrix}$

For the uniform de-quantization process in equation 20, the optimalquantized coefficient can be obtained as equation 24.

ĉ _(i)=λ_(i)round(c _(i)*/λ_(i))=U(ε_(i) ,g _(i),λ_(i))  [Equation 24]

This may lead to the quantization algorithm to be explained hereafter.

The encoder may perform repetitive simulations in order to obtain anoptimized coefficient to be transmitted to the decoder at step S810.

If a current coefficient satisfies a specific condition as a result of acomparison between the current coefficient and a previous coefficient,the current coefficient may be determined to be an optimizedcoefficient. For example, assuming that the current coefficient is C_(n)and the previous coefficient is C_(n-1), whether a difference valueC_(n-1)−C_(n) between the current coefficient and the previouscoefficient converges on 0 may be checked at step S820. If, as a resultof the check, the difference value C_(n-1)−C_(n) is found to converge on0, the current coefficient C_(n) may be determined to be an optimizedcoefficient and transmitted to the decoder at step S830. If, as a resultof the check, the difference value C_(n-1)−C_(n) is found to notconverge on 0, the current coefficient C_(n) may be returned so that theprevious steps S810 and S820 are repeatedly performed.

In another specific condition, an optimized coefficient may bedetermined by comparing the difference value C_(n-1)−C_(n) between thecurrent coefficient and the previous coefficient with a specificthreshold τ. For example, if, as a result of the comparison, thedifference value C_(n-1)−C_(n) is found to be greater than the specificthreshold τ, the current coefficient C_(n) may be returned so that theprevious steps S810 and S820 are repeatedly performed. In contrast, if,as a result of the comparison, the difference value C_(n-1)−C_(n) isfound to be equal to or smaller than the specific threshold τ, thecurrent coefficient C_(n) may be determined to be an optimizedcoefficient and transmitted to the decoder.

Such an operation may be performed by the encoder of FIG. 3. Forexample, the operation may be performed by the optimizer 310.

FIG. 9 illustrates an embodiment to which the present invention may beapplied and is a detailed flowchart illustrating a quantization processfor obtaining an optimized coefficient.

In accordance with an embodiment of the present invention, the encodermay obtain an optimized coefficient based on at least one of informationabout the pixels of a current block, information about the pixels of apreviously decoded block, and information about a quantization stepsize. Such an operation may be performed by the quantization unit of theencoder.

First, the encoder may obtain an initially quantized coefficient basedon information about the pixels of a current block and information aboutthe pixels of a previously decoded block at step S910. The initiallyquantized coefficient may be represented as in equation 25.

C ₀ =G ⁻¹(x−Hy)  [Equation 25]

In this case, C₀ denotes an initially quantized coefficient, x denotesinformation about the pixels of a current block, and y denotesinformation about the pixels of a previously decoded block. In thiscase, G, H denotes matrices optimized on training sets. Furthermore, thematrix G may be indicative of a non-orthogonal transform matrix.

An error vector indicative of a difference between the original signaland a reconstructed signal may be obtained based on the initiallyquantized coefficient at step S920. In this case, the pixel informationx of the current block and the pixel information y of the previouslydecoded block may be used, which may be represented as in equation 26.

e _(n) =x−Hy−GC _(n-1)  [Equation 26]

In equation 26, e_(n) denotes an error vector, and n=0, 1, 2, . . . ,which may be repeatedly performed until an optimized coefficient isobtained. For such an iteration process, a temporary vector may bedefined as in equation 27.

t=e _(n) +g _(k) C _(n-1)(k)  [Equation 27]

In equation 27, t denotes a temporary vector, and g_(k) denotes k^(th)column vector of a matrix G. Furthermore, C_(n-1) (k) denotes a(n−1)^(th) quantized coefficient.

An n^(th) quantized coefficient C_(n) may be obtained based on thetemporary vector t and quantization step size information λ(k) at stepS930. In this case, equation 28 may be used.

C _(n)(k)=λ(k)round(g _(k) ^(T) t/λ(k)(g _(k) ^(T) g _(k))) (k=1,2, . .. ,B ²)  [Equation 28]

In equation 28, λ(k) denotes a quantization step size that is to be usedfor a k^(th) transform coefficient.

Furthermore, the error vector e_(n) may be updated as in equation 29 atstep S940.

e _(n) +=g _(k)(C _(n-1)(k)−C _(n)(k))  [Equation 29]

If the n^(th) quantized coefficient C_(n) is obtained through such aprocess, whether a specific condition is satisfied may be checked bycomparing the n^(th) quantized coefficient C_(n) with the previouscoefficient C_(n-1). The n^(th) quantized coefficient C_(n) may bedetermined to be an optimized coefficient based on a result of thecomparison. For example, whether a difference value C_(n-1)−C_(n)between the n^(th) quantized coefficient C_(n) and the previouscoefficient C_(n-1) converges on 9 may be checked at step S950.

If, as a result of the check, the difference value C_(n-1)−C_(n) isfound to converge on 0, the n^(th) quantized coefficient C_(n) may bedetermined to be an optimized coefficient and transmitted to the decoderat step S960. In contrast, if, as a result of the check, the differencevalue C_(n-1)−C_(n) is found to not converge on 0, the n^(th) quantizedcoefficient C_(n) may be returned so that the previous steps areiterated.

In yet another specific condition, an optimized coefficient may bedetermined by comparing a difference value C_(n-1)−C_(n) between acurrent coefficient and a previous coefficient with a specific thresholdτ. For example, this may be represented as in equation 30.

∥C _(n) −C _(n-1)∥₂>τ  [Equation 30]

If a difference value ∥C_(n)−C_(n-1)∥₂ is greater than a specificthreshold τ, the current coefficient C_(n) may be returned so thatprevious steps are iterated. In contrast, if the difference value∥C_(n)−C_(n-1)∥₂ is equal to or smaller than the specific threshold τ,the current coefficient C_(n) may be determined to be an optimizedcoefficient and transmitted to the decoder.

FIG. 10 illustrates an embodiment to which the present invention may beapplied and is a flowchart illustrating a process of obtaining anoptimized quantization step size.

As described above with reference to FIG. 9, an optimized quantizationstep size may be derived in a process of performing, by the encoder,quantization in order to obtain an optimized coefficient.

First, quantization step size information may be obtained from aquantization parameter value at step S1010. For example, thequantization step size information may be represented as in equation 31.

Δ(k)=2^((QP-4)/6) (k=1,2, . . . ,B ²)  [Equation 31]

In equation 31, Δ(k) denotes a k^(th) quantization step size, and QPdenotes a quantization parameter.

Matrices and a vector to be used to obtain an optimized coefficient maybe initialized at step S1020. For example, the vector and the matricesmay be represented as in equations 32 and 33.

u(k)=1 (k=1,2, . . . ,B ²)  [Equation 32]

G (k,l)=G(k,l)²(l=1,2, . . . ,B ²)

H (k,l)=H(k,l)²(l=1,2, . . . ,B ²)  [Equation 33]

The optimizer may obtain an optimized quantization step size based onthe k^(th) quantization step size Δ(k) and the initialized vector u(k)and matrices G(k,l),H(k,l) at step S1030. In this case, a convexoptimization algorithm may be used.

An embodiment of the present invention can provide a method of derivingoptimal quantizer step-sizes.

Rate-Distortion optimal design of quantizer step-sizes is in general adifficult problem since tractable expressions for rate and distortionare codec dependent and hard to obtain. In this embodiment, the highrate approximations can be used in order to optimize the vector ofstep-sizes, λ.

The transform coding recipe followed by successful image and videocoders utilize scalar entropy coders. Thus the rate required to conveythe quantized coefficients in Ĉ can be approximated as equation 34.

$\begin{matrix}{{\left( \hat{c} \right)} = {\sum\limits_{k}{H\left( {\hat{c}}_{k} \right)}}} & \left\lbrack {{Equation}\mspace{14mu} 34} \right\rbrack\end{matrix}$

In equation 34, H( ) denotes entropy. Since coefficient Ĉ_(i) is scalarquantized using the step-size λ_(i), the approximation can be invoked athigh bit-rates.

H(ĉ _(i))≈h(c _(i))−log(λ_(i))  [Equation 35]

In equation 35, h(c_(i)) is the differential entropy of the continuouslyvalued coefficient. Hence, in order to meet a rate constraint, equation36 may be needed.

$\begin{matrix}{{\sum\limits_{k}{\log \left( \lambda_{k} \right)}} \sim {constant}} & \left\lbrack {{Equation}\mspace{14mu} 36} \right\rbrack\end{matrix}$

Had G been orthonormal, a straightforward approximation for averagedistortion in terms of λ would have been

_(orth)(λ)=Σ_(k)λ_(k) ²/12, which is obtained by assuming uniformlydistributed quantization error.

With a non-orthogonal G, signal domain and coefficient domaindistortions are not the same and one cannot use this approximation.Assume all quantities are zero mean. The signal domain averagedistortion can be written as where E[ ] denotes expectation and Tr(.) isthe trace of a matrix. Using e=G(c−Ĉ), equation 37 can be obtained.

$\begin{matrix}\begin{matrix}{{(\lambda)} = {{Tr}\left( {{{GE}\left\lbrack {\left( {c - \hat{c}} \right)\left( {c - \hat{c}} \right)^{T}} \right\rbrack}G^{T}} \right)}} \\{= {{Tr}\left( {{{GE}\left\lbrack {pp}^{T} \right\rbrack}G^{T}} \right)}}\end{matrix} & \left\lbrack {{Equation}\mspace{14mu} 37} \right\rbrack\end{matrix}$

In equation 37, p=c−Ĉ has been set to denote the coefficient domainerror. Assuming that coefficient domain error is decorrelated, i.e.,E[pp^(T)] is diagonal with diagonal entries n_(i), i=1, . . . N,straightforward algebra yields equation 38.

$\begin{matrix}{{(\lambda)} = {\sum\limits_{k = 1}^{N}{\sum\limits_{l = 1}^{N}{G_{k,l}^{2}\pi_{l}}}}} & \left\lbrack {{Equation}\mspace{14mu} 38} \right\rbrack\end{matrix}$

Since the quantization is carried out through the quantizationalgorithm, approximations of the form π=λ_(i) ²/12 are not valid. Inorder to relate π to λ let us concentrate on the rounding error inducedby the quantization algorithm. At the point of convergence, equation 39can be obtained.

$\begin{matrix}\begin{matrix}{\iota_{k} = {{{\hat{c}}_{k}/\lambda_{k}} = {{round}\left( \frac{g_{k}^{T}\left( {e + {g_{k}{\hat{c}}_{k\;}}} \right)}{\left( {g_{k}^{T}g_{k}} \right)\lambda_{k}} \right)}}} \\{{= {{{round}\left( \frac{g_{k}^{T}e}{\left( {g_{k}^{T}g_{k}} \right)\lambda_{k}} \right)} + \iota_{k}}},}\end{matrix} & \left\lbrack {{Equation}\mspace{14mu} 39} \right\rbrack\end{matrix}$

Equation 39 leads to the rounding error satisfying

${\frac{g_{k}^{T}e}{\left( {g_{k}^{T}g_{k}} \right)\lambda_{k}}} < {0.5.}$

Set

$\omega_{k} = \frac{g_{k}^{T}e}{\left( {g_{k}^{T}g_{k}} \right)}$

and if we assume that the rounding error is uniform, equation 40 can beobtained.

E[(g _(k) ^(T) e/(g _(k) ^(T) g _(k)))² ]=E[ω _(k) ²]≅λ_(k)²/12  [Equation 40]

Let Ĝ be the matrix with the i^(th) column

$\frac{g_{i}}{\left( {g_{i}^{T}g_{i}} \right)}.$

Equation 41 can be obtained.

ω= G ^(T) e=G ^(T) G(c−{circumflex over (c)})  [Equation 41]

Letting H=Ĝ^(T)G, Equation 42 can be obtained.

$\begin{matrix}\begin{matrix}{{{{HE}\left\lbrack {\left( {c - \hat{c}} \right)\left( {c - \hat{c}} \right)^{T}} \right\rbrack}H^{T}} = {{{HE}\left\lbrack {pp}^{T} \right\rbrack}H^{T}}} \\{= {{E\left\lbrack {\omega \; \omega^{T}} \right\rbrack}.}}\end{matrix} & \left\lbrack {{Equation}\mspace{14mu} 42} \right\rbrack\end{matrix}$

Considering the diagonal elements of Equation 42 may lead to Equation43.

$\begin{matrix}{{\sum\limits_{l = 1}^{N}{H_{k,l}^{2}\pi_{l}}} = {{E\left\lbrack \omega_{k}^{2} \right\rbrack} \simeq {\lambda_{k}^{2}/12}}} & \left\lbrack {{Equation}\mspace{14mu} 43} \right\rbrack\end{matrix}$

Let G and H denote the matrices that have the matrix elements squared ofG and H respectively. Equations 38 and 43 become equation 44.

(λ)=u ^(T) Gπ

Hπ=λ/12  [Equation 44]

In equation 44, u is the vector of all-ones and λ _(i)=λ_(i) ².

Accordingly, equation 45 can be obtained.

$\begin{matrix}{{{(\lambda)} = {a^{T}\overset{\_}{\lambda}}},{{{where}\mspace{14mu} a^{T}} = \frac{u^{T}{GA}^{- i}}{12}}} & \left\lbrack {{Equation}\mspace{14mu} 45} \right\rbrack\end{matrix}$

The optimization can be put in the form of the minimization of averagedistortion (equation 45) subject to the rate constraint to obtainequation 46.

$\begin{matrix}{\min\limits_{\lambda}\left\{ {{\sum\limits_{k}{a_{k}\lambda_{k}^{2}}} + {\gamma {\sum\limits_{k}{\log \left( \lambda_{k}^{2} \right)}}}} \right\}} & \left\lbrack {{Equation}\mspace{14mu} 46} \right\rbrack\end{matrix}$

In equation 46, γ is a Lagrange multiplier. Optimization of equation 46yields the following equation 47.

λ_(i)=√{square root over (γ/a _(i))}  [Equation 47]

FIGS. 11 and 12 illustrate embodiments to which the present inventionmay be applied, wherein FIG. 11 illustrates test images to which thepresent invention has been applied and FIG. 12 illustrates percentagesof rate gains to test images.

As described above, in accordance with an embodiment of the presentinvention, a signal having an edge and directional structure can becoded more efficiently by non-casually predicting a video signal using atransform-coded signal along with a predicted signal.

In the present simulation, intra prediction was performed on a layer ofa 1-pixel thickness within a block, and the prediction process and thequantization process described with reference to FIGS. 3 to 10 wereapplied to the simulation.

FIG. 11 illustrates 6 test images (a)-(f), and each of the 6 images hasan image feature.

Each of the 6 test images may be considered to correspond to a signal inwhich at least one of an edge and directional singularity significantlyappears compared to other common images.

As a result of measuring the rate gains of such test images, results,such as those of FIG. 12(a), can be found. That is, from FIG. 12(a), itmay be seen that effects have been improved compared to efficiency of anexisting codec with respect to all the 6 test images.

It may also be seen that the images FIGS. 11(a), 11(b), and 11(e) havesignificant directional singularities compared to the remaining imagesFIGS. 11(c), 11(d), and 11(f). Accordingly, from FIG. 12(a), it may beseen that the images FIGS. 11(a), 11(b), and 11(e) have relativelyhigher rate gains.

Likewise, from FIG. 12(b), it may be seen that in the case of thesimulation for a video sequence, effects have been improved compared toefficiency of an existing codec.

FIG. 13 illustrates an embodiment to which the present invention may beapplied and is a schematic flowchart illustrating an improved predictivecoding method.

First, when the original video signal is received at step S1310, theencoder may compare the original video signal with availablereconstructed signals at step S1320. And, the encoder may determine acorrection signal based on a result of the comparison.

In this case, the correction signal may be determined to minimize a sumof a distortion component and a rate component. The distortion componentis indicative of total distortion between the original video signal andthe correction signal, and the rate component is indicative of a numberof bits required to send the transform-coded correction signal. In orderto determine a correction signal, the encoder may perform decodingsimulations.

The encoder may generate a transform-coded correction signal based on aresult of the comparison at step S1330.

And, the encoder may generate a prediction signal based on thetransform-coded correction signal and the available reconstructedsignals at step S1340.

And then, the encoder may reconstruct a signal by adding thetransform-coded correction signal to the prediction signal at stepS1350.

FIG. 14 illustrates an embodiment to which the present invention may beapplied and is a schematic flowchart illustrating a method of performingquantization based on an optimized quantization step size.

An embodiment of the present invention provides a method of deriving anoptimized quantization step size in a process of performing quantizationin order to obtain an optimized coefficient. Quantization may beperformed based on the derived quantization step size.

First, information about a quantization step size may be obtained from aquantization parameter value. In this case, the quantization step sizeinformation may mean a scaling parameter. The scaling parameter may beobtained using a Rate Distortion (RD)-optimized algorithm. For example,the scaling parameter may be determined to be a value that minimizes thesum of a distortion component and a rate component at step S1410.

A transform-coded correction signal may be obtained according to theembodiments described above with reference to FIGS. 8 to 10. Forexample, the transform-coded correction signal may include an optimizedtransform coefficient.

At step S1420, quantization may be performed on the transform-codedcorrection signal based on the scaling parameter determined at stepS1410.

The quantized coefficient may be subjected to entropy encoding andtransmitted at step S1430.

FIGS. 15 and 16 illustrate embodiments to which the present inventionmay be applied and are schematic block diagrams of an encoder and adecoder to which an advanced coding method has been applied throughcontrol of a quantization error.

The present invention defines a set of coding parameter to controlquantization effects by manipulating factors simultaneously in threespaces: spatial, spectral, and lattice norm. Improved compression may beprovided by finding optimized parameters determined using a specifictype and training technology of an image compression method.

In FIG. 1, all the factors required for predictive coding, transformcoding, and hybrid coding are included.

Predictive coding is based on that a signal element is predicted using apreviously coded part and a difference value between a predicted valueand an actual value is coded. An N-dimensional vector X is used toindicate coded data (e.g., an image or video frame), and a vector P isused to indicate a value predicted from the N-dimensional vector X. Suchprediction is performed using a vector y formed from the past values ofa reconstructed vector {tilde over (X)}.

First, a difference vector indicative of a prediction residual may becomputed as in the following equation 48.

d=x−p(y)  [Equation 48]

In general, such a difference is additionally transformed usingorthogonal linear transform represented by an N×N matrix T. Thereafter,a vector coefficient is converted into an integer for entropy coding.

A vector having an integer coefficient is indicated by c and may bedefined as in the following equation 49.

c=Q(T[x−p]),ci∈Z,i=1,2, . . . ,N  [Equation 49]

In general, quantization is performed using an orthogonal scaling matrixQ and may be defined as in the following equation 50.

c=[[QT(x−p)]]  [Equation 50]

In equation 50, double brackets [[ ]] denotes per-element rounding as inthe following equation 51.

c=[[v]]

c _(i) =└v _(i)+0.5┘i=1,2, . . . ,N  [Equation 51]

The reconstructed vector {tilde over (X)} may be computed by both theencoder and the decoder using the following equation 52.

{tilde over (X)}=p+T ⁻¹ Q ⁻¹ c  [Equation 52]

In equation 52, {tilde over (X)} denotes a reconstructed vector, pdenotes a prediction vector, T denotes a transform matrix, Q denotes aquantization matrix, and c denotes a transform coefficient.

If the matrix T is defined by transform, such as DCT, the application ofsuch transform is almost the same as that the spectral component of aresidual vector d is computed. Accordingly, in an embodiment of thepresent invention, the distribution of quantization errors in afrequency domain may be changed using different values of a diagonalmatrix Q.

All the elements within the vector of an image or video block may not beused in the same way when inter block prediction is performed.Accordingly, prediction precision may be significantly reduced due tothe errors of some elements present at the boundary of a block.

Furthermore, if linear transform, such as DCT, is independently appliedto a vector, a blocking artifact may be generated at the boundary of ablock.

In this case, importance is different in each of errors occurring inother parts of a block. In an embodiment of the present invention, ablocking artifact can be reduced by providing a method of controlling aquantization error more finely on a space domain. However, an approachusing a diagonal matrix Q may be controlled only in a frequency domain.Accordingly, the present invention can solve such a problem bycontrolling a quantization error in both the space domain and thefrequency domain.

Referring to FIG. 15, the encoder 1500 to which the present inventionmay be applied may include an optimizer 1520, a dequantization unit1530, an inverse transform unit 1540, a buffer 1550, a prediction unit1560, and an entropy encoding unit 1570. In this case, the inversetransform unit 1540 may include a spatial scaling unit 1545.

Referring to the encoder 1500 of FIG. 15, the optimizer 1520 may obtainan optimally quantized transform coefficient.

First, the optimizer 1520 may obtain an optimally quantized transformcoefficient through a training step. For example, the optimizer 1520 maycompute an optimized set of diagonal matrices S, W, and Q from aviewpoint of Rate Distortion (RD).

An embodiment of the present invention provides a method of addinganother diagonal matrix S, that is, a scaling factor on a space domain.In such a case, a process for reconstructing a signal may be changed asin the following equation 53.

x=p+ST ⁻¹ Q ⁻¹ c  [Equation 53]

An orthogonal condition on which an optimized transform coefficient iscalculated using simple rounding as in Equation 3 may be changed.Accordingly, in an embodiment of the present invention, an optimizedtransform coefficient may be calculated based on the following equation54.

$\begin{matrix}{{c = {\underset{a \in {\mathbb{Z}}^{N}}{\arg \; \min}\left\{ {_{W}\left( {x - p - {{ST}^{- 1}Q^{- 1}a}} \right)} \right\}}}{{where}\mspace{14mu} {D_{W}(v)}} = {v^{T}W^{2}v}} & \left\lbrack {{Equation}\mspace{14mu} 54} \right\rbrack\end{matrix}$

In Equation 54, W denotes another diagonal matrix used to differentiatethe importance of errors in the spatial domain.

Furthermore, in an embodiment of the present invention, in order to findan optimized set of the diagonal matrices S, W, and Q, objectivedistortion measurement, such as a Mean Squared Error (MSE), and anotherdistortion measurement including subjective factors, such as thevisibility of blocking artifacts, may be used.

Furthermore, prior to the coding of an image or video frame, the valuesof diagonal matrices S, W, and Q, that is, side information, may beencoded. In this case, a proper protocol that may be recognized by thedecoder may be used.

The dequantization unit 1530 may obtain a transform coefficient byperforming dequantization on the optimally quantized transformcoefficient.

The inverse transform unit 1540 may obtain a predicted error vector byperforming inverse transform on the transform coefficient. In this case,the inverse transform may include a scale orthogonal matrix S. By addingthe scaling matrix on the space domain as described above, aquantization error can be controlled even on the space domain.

Scaling using the scale orthogonal matrix S may be performed by thespatial scaling unit 1545 of the inverse transform unit 1540.Furthermore, the spatial scaling unit 1545 may be placed after theinverse transform process of the inverse transform unit 1540.

A reconstructed signal may be generated by adding the obtained predictederror vector to a prediction signal output by the prediction unit 1560.

The buffer 1550 stores the reconstructed signal for the future referenceof the prediction unit 1560. The prediction unit 1560 generates aprediction signal using a previously reconstructed signal stored in thebuffer 1550.

The optimally quantized transform coefficient obtained by the optimizer1520 may be transmitted to the entropy encoding unit 1570. The entropyencoding unit 1570 may perform entropy encoding on the optimallyquantized transform coefficient and output the resulting transformcoefficient.

Referring to FIG. 16, the decoder 1600 to which the present inventionmay be applied may include an entropy decoding unit 1610, adequantization unit 1620, an inverse transform unit 1630, a buffer 1640,and a prediction unit 1650. In this case, the inverse transform unit1630 may include a spatial scaling unit 1635.

The decoder 1600 of FIG. 16 receives a signal output by the encoder 1500of FIG. 15. The received signal is subjected to entropy decoding throughthe entropy decoding unit 1610.

The dequantization unit 1620 obtains a transform coefficient from theentropy-decoded signal using quantization step size information. Theinverse transform unit 1630 obtains a predicted error by performinginverse transform on the transform coefficient. In this case, theinverse transform may include a scale orthogonal matrix S.

Scaling using the scale orthogonal matrix S may be performed by thespatial scaling unit 1635 of the inverse transform unit 1630. Thespatial scaling unit 1635 may be placed after the inverse transformprocess of the inverse transform unit 1630. Furthermore, the embodimentsdescribed with reference to FIG. 15 may be applied.

A reconstructed signal is generated by adding the obtained predictederror to a prediction signal output by the prediction unit 1650.

The buffer 1640 stores the reconstructed signal for the future referenceof the prediction unit 1650. The prediction unit 1650 may generate aprediction signal using a previously reconstructed signal stored in thebuffer 1640.

FIG. 17 illustrates an embodiment to which the present invention may beapplied and is a flowchart illustrating a process of obtaining a scalingdiagonal matrix through a Rate Distortion (RD) optimization process.

In the form of coding defined by equation 46, an approximatedreproduction of pixels values can be obtained because c∈Z^(N), i.e., thevector of data to be coded can only have integer values.

The present invention can model the approximation using statisticalmethods, by defining an additive error vector e.

T ⁻¹ Q ⁻¹ c=x−p+e  [Equation 55]

The rounding errors in each component of c can be combined throughmultiplication by the orthogonal matrix T. For high-rate approximations,we can assume that components of e are independent random Gaussianvariables, with zero mean and same variance. Thus, the values obtainedfrom equation 46 yields equation 56.

{tilde over (X)}=p+T ⁻¹ Q ⁻¹ c=x+e  [Equation 56]

Equation 56 means that errors have roughly the same distribution for allpixels in a block.

With the new approach defined by equation 57, the residual values x−ppre-scaled by S−1 to obtain the proper reproduction.

ST ⁻¹ Q ⁻¹ c=S[S ⁻¹(x−p)+e]  [Equation 57]

The elements of e are independent random Gaussian variables, with zeromean and same variance.

However, in this case we have reproduced pixels given by equation 58.

{tilde over (X)}=p+ST ⁻¹ Q ⁻¹ c=x+Se  [Equation 58]

Equation 52 means that now the error in each pixel has differentvariances, proportional to the scaling factors in diagonal matrix S.Larger values of S_(i,i) thus produce relatively larger error variances,and vice-versa.

In the following we present a somewhat more detailed description of theinvention. The present invention can be applied for each pre-definedvideo segment, for example, coding unit, frame, tile, slice, etc.

At the encoder, the present invention can be performed according to thefollowing steps.

First, the encoder can choose the matrices S, W, and Q to be used forcoding pixel blocks within the segment.

Next, before coding pixels in each segment, the encoder can add to thecompressed bitstream the information about matrices S and Q. Forexample, T is assumed constant, and W is only used by the encoder.

And then, for each pixel block, the encoder can find the optimal vectorc∈Z^(N), entropy code its value, and add it to the compressed bitstream.

At the decoder, the present invention can be performed according to thefollowing steps.

First, before decoding pixels in each segment, the decoder can read fromthe compressed bitstream the information about matrices S and Q.

And then, for each pixel block, the decoder can entropy decode thevector c∈Z^(N), and compute reconstructed pixel values using equation59.

{tilde over (X)}=p+ST ⁻¹ Q ⁻¹ c  [Equation 59]

An embodiment of the present invention provides a process of obtaining ascaling diagonal matrix through a Rate Distortion (RD) optimizationprocess.

First, the encoder may perform an RD optimization process throughtraining at step S1710. For example, such an RD optimization process maybe performed by the optimizer 1520.

An optimized set of diagonal matrices S, W, and Q may be computedthrough the RD optimization process at step S1720.

The values of the diagonal matrices S, W, and Q may be encoded into sideinformation at step S1730.

Thereafter, a video signal may be coded or decoded according to theprocesses described with reference to FIGS. 15 and 16 at step S1740.

For example, the scaling diagonal matrix S of the diagonal matrices maybe used in the inverse transform unit 1540 of the encoder 1500 or theinverse transform unit 1630 of the decoder 1600 so that a quantizationis controlled error even on the space domain.

FIG. 18 illustrates an embodiment to which the present invention may beapplied and is a graph illustrating a comparison between the codinggains of respective images in the case in which coding is performedusing an optimized scaling matrix and the case in which coding isperformed using an existing method.

FIG. 18 illustrates relations between control of error transfer andcoding gains.

Dotted lines in the graph denote the coding gains of a common codec, andsolid lines denote coding gains when optimized diagonal matrices areused.

The present embodiment corresponds to a case where planar prediction and4×4 DCT are used. It may be seen that better coding efficiency isobtained when all the optimized diagonal matrices are used in three testimages, “Woman”, “Bike”, and “Café”.

This is only an embodiment of the present invention, and the presentinvention is not limited to the aforementioned conditions and may beapplied to embodiments having other conditions.

FIGS. 19 and 20 are embodiments to which the present invention may beapplied and are schematic block diagrams illustrating an encoder and adecoder to which an advanced coding method may be applied.

The encoder 1900 of FIG. 19 includes an optimizer 1910, a quantizationunit 1920, and an entropy encoding unit 1930. The decoder 2000 of FIG.20 includes an entropy decoding unit 2010, a dequantization unit 2020,an inverse transform unit 2030, and a reconstruction unit 2040.

Referring to the encoder 1900 of FIG. 19, the optimizer 1910 obtains anoptimized transform-coded correction signal. The optimizer 1910 may usethe following embodiments in order to obtain the optimizedtransform-coded correction signal.

In order to illustrate an embodiment to which the present invention maybe applied, first, a reconstruction function for reconstructing a signalmay be defined as follows.

{tilde over (x)}=R(c,y)  [Equation 60]

In Equation 60, {tilde over (x)} denotes a reconstructed signal, cdenotes a decoded transform-coded correction signal, and y denotes acontext signal. R(c,y) denotes a reconstruction function using c and yin order to generate a reconstructed signal.

In the present embodiment, a reconstruction function may be defined as arelationship between previously reconstructed values and atransform-coded correction signal. Accordingly, the decoded correctionsignal affects not only the reconstruction value, but also the entirereconstruction process and the choice of reconstruction functions.

For example, a correction signal may be defined as follows.

e=Tc  [Equation 61]

In Equation 61, e denotes a correction signal, c denotes atransform-coded correction signal, and T denotes a transform matrix.Also, in some cases, the correction signal may mean error signal orprediction error signal.

In this case, a reconstructed signal may be defined as follows.

$\begin{matrix}{{{{\overset{\sim}{x}}_{1} = {R_{1}\left( {e,y} \right)}}{\overset{\sim}{x}}_{2} = {R_{2}\left( {e,y,{\overset{\sim}{x}}_{1}} \right)}}\vdots {{\overset{\sim}{x}}_{n} = {R_{n}\left( {e,y,{\overset{\sim}{x}}_{1},{{\overset{\sim}{x}}_{2}\mspace{14mu} \ldots}\mspace{14mu},{\overset{\sim}{x}}_{n - 1}} \right)}}} & \left\lbrack {{Equation}\mspace{14mu} 62} \right\rbrack\end{matrix}$

In Equation 62, {tilde over (x)}_(n) denotes an n^(th) component of thereconstructed signal, e denotes the correction signal, and y denotes acontext signal. R_(n) denotes a reconstruction function using e, y and{tilde over (x)} in order to generate a reconstructed signal.

In one embodiment, the reconstruction function R_(n) may be defined asfollows.

$\begin{matrix}{\mspace{79mu} {{{R_{1}\left( {e_{1},y} \right)} = {{P_{1}(y)} + e_{1}}}\mspace{79mu} {{R_{2}\left( {e_{2},y,{\overset{\sim}{x}}_{1}} \right)} = {{P_{2}\left( {y,{\overset{\sim}{x}}_{1}} \right)} + e_{2}}}}} & \left\lbrack {{Equation}\mspace{14mu} 63} \right\rbrack \\{\mspace{85mu} \vdots} & \; \\{{R_{n}\left( {e_{2},y,{\overset{\sim}{x}}_{1},\ldots \mspace{14mu},{\overset{\sim}{x}}_{n - 1}} \right)} = {{P_{n}\left( {y,{\overset{\sim}{x}}_{1},{{\overset{\sim}{x}}_{2}\mspace{14mu} \ldots}\mspace{14mu},{\overset{\sim}{x}}_{n - 1}} \right)} + e_{n}}} & \;\end{matrix}$

In Equation 63, P_(n) denotes a type of prediction function formed ofthe parameters in order to generate a prediction signal.

The prediction function may be, for example, a median function, acombination of a rank order filter and a non-linear function, or acombination of linear functions. Furthermore, each of the non-linearprediction function P_(n)( ) may be a different non-linear function.

In another embodiment of the present invention, a quantization unit 1920may be included in the optimizer 1910, or the optimizer 1910 may includetransform unit.

In another embodiment of the present invention, the encoder 1900 and thedecoder 2000 may include a storage unit of candidate functions forselecting the non-linear prediction function.

In this case, the optimized non-linear prediction function may beselected from candidate functions stored in the storage unit.

When an optimized non-linear prediction function is selected asdescribed above, the optimizer 1910 may generate an optimized predictionsignal using the optimized non-linear prediction function. And, theoptimizer 1910 may generate an optimized prediction error signal basedon the optimized prediction signal, and may perform transform coding onthe optimized prediction error signal. The optimizer 1910 may output atransform-coded coefficient through the transform coding. In this case,the transform-coded coefficient may mean an optimized transformcoefficient.

The output transform coefficient is transmitted to the quantization unit1920. The quantization unit 1920 quantizes the transform coefficient andsends the quantized transform coefficient to the entropy encoding unit1930.

The entropy encoding unit 1930 may perform entropy encoding on thequantized transform coefficient and output a compressed bit stream.

The decoder 2000 of FIG. 20 may receive the compressed bit stream fromthe encoder of FIG. 19, may perform entropy decoding through the entropydecoding unit 2010, and may perform dequantization through thedequantization unit 2020. In this case, a signal output by thedequantization unit 2020 may mean an optimized transform coefficient.

The inverse transform unit 2030 receives the optimized transformcoefficient, performs an inverse transform process, and may obtain aprediction error signal through the inverse transform process.

The reconstruction unit 2040 may obtain a reconstructed signal by addingthe prediction error signal and a prediction signal together. In thiscase, various embodiments described with reference to FIG. 19 may beapplied to the prediction signal.

FIG. 21 is an embodiment to which the present invention may be appliedand is a schematic flowchart illustrating an advanced video codingmethod.

First, when the original video signal is received at step S2110, theencoder may compare the original video signal with availablereconstructed signals at step S2120. And, the encoder may determine acorrection signal based on a result of the comparison at step S2130.

In this case, the correction signal may be determined to minimize a sumof a distortion component and a rate component. The distortion componentis indicative of total distortion between the original video signal andthe correction signal, and the rate component is indicative of a numberof bits required to send the transform-coded correction signal. In orderto determine a correction signal, the encoder may perform decodingsimulations.

This invention may further comprise determining a reconstructionfunction to be used for the signal reconstruction, and thereconstruction function includes at least one of a linear component anda non-linear component.

And, the reconstruction function may be determined based on all thepreviously reconstructed samples and the correction signal.

And then, the encoder may generate a transform-coded correction signalto be transmitted for a signal reconstruction at step S2140. Here, thetransform-coded correction signal may be multiplied by a dequantizationmatrix and an inverse-transform matrix, and wherein the dequantizationmatrix may be selected for controlling a bit-rate and quantizationerrors.

Furthermore, the transform-coded correction signal may correspond to thecorrection signal for a group of pictures and a spatiotemporal transformcoding may has been applied to the correction signal.

In accordance with an embodiment of the present invention, the decodermay receive a bit stream including a transform-coded correction signalobtained according to the present invention, may perform entropydecoding through the entropy decoding unit, may perform dequantizationthrough the dequantization unit, and may perform inverse transformthrough the inverse transform unit. The decoder may obtain a correctionsignal by performing inverse-transform to the transform-coded correctionsignal.

And then the decoder may obtain a reconstructed signal using areconstruction function that combines the obtained correction signal anda context signal. Here, the context signal may be obtained based on allpreviously reconstructed samples.

Furthermore, the decoder may determine a reconstruction function to beused for the signal reconstruction, and the reconstruction function mayinclude at least one of a linear component and a non-linear component.Here, the reconstruction function may be determined based on all thepreviously reconstructed samples and the correction signal.

The transform-coded correction signal may be multiplied by adequantization matrix and an inverse-transform matrix. Also, thetransform-coded correction signal may correspond to the correctionsignal for a group of pictures and a spatiotemporal transform coding hasbeen applied to the correction signal.

As described above, the decoder and the encoder to which the presentinvention may be applied may be included in a multimedia broadcastingtransmission/reception apparatus, a mobile communication terminal, ahome cinema video apparatus, a digital cinema video apparatus, asurveillance camera, a video chatting apparatus, a real-timecommunication apparatus, such as video communication, a mobile streamingapparatus, a storage medium, a camcorder, a VoD service providingapparatus, an Internet streaming service providing apparatus, athree-dimensional (3D) video apparatus, a teleconference videoapparatus, and a medical video apparatus and may be used to code videosignals and data signals.

Furthermore, the decoding/encoding method to which the present inventionmay be applied may be produced in the form of a program that is to beexecuted by a computer and may be stored in a computer-readablerecording medium. Multimedia data having a data structure according tothe present invention may also be stored in computer-readable recordingmedia. The computer-readable recording media include all types ofstorage devices in which data readable by a computer system is stored.The computer-readable recording media may include a BD, a USB, ROM, RAM,CD-ROM, a magnetic tape, a floppy disk, and an optical data storagedevice, for example. Furthermore, the computer-readable recording mediaincludes media implemented in the form of carrier waves (e.g.,transmission through the Internet). Furthermore, a bit stream generatedby the encoding method may be stored in a computer-readable recordingmedium or may be transmitted over wired/wireless communication networks.

INDUSTRIAL APPLICABILITY

The exemplary embodiments of the present invention have been disclosedfor illustrative purposes, and those skilled in the art may improve,change, replace, or add various other embodiments within the technicalspirit and scope of the present invention disclosed in the attachedclaims.

1. A method of encoding a video signal, comprising: receiving anoriginal video signal; comparing the original video signal with apreviously reconstructed signal; generating a correction signal tominimize a sum of a distortion component and a rate component; andentropy-encoding the correction signal that is transmitted to thedecoder for video signal reconstruction, wherein the previouslyreconstructed signal has been inverse-transformed by additionally usinga scaling diagonal matrix.
 2. The method of claim 1, wherein thecorrection signal is generated based on another diagonal matrix beingused to differentiate a weighting of errors in a spatial domain.
 3. Themethod of claim 1, further comprising: calculating an optimal set ofmultiple diagonal matrices including the scaling diagonal matrix,wherein the correction signal is generated based on the optimal set ofmultiple diagonal matrices.
 4. The method of claim 3, wherein theoptimal set of multiple diagonal matrices is encoded as sideinformation, and is transmitted to a decoder.
 5. The method of claim 3,wherein the optimal set of multiple diagonal matrices is encoded beforeencoding frames of the original video signal.
 6. The method of claim 1,wherein: the distortion component is indicative of total distortionbetween the original video signal and a reconstructed signal, and therate component is indicative of a number of bits required to send aquantized coefficient.
 7. A method of decoding a video signal,comprising: receiving the video signal including a correction signal;reading side information including multiple diagonal matrices from thevideo signal; obtaining the correction signal by entropy-decoding thevideo signal; and reconstructing a signal based on the correction signaland the multiple diagonal matrices.
 8. The method of claim 7, whereinthe multiple diagonal matrices includes a scaling diagonal matrix. 9.The method of claim 8, further comprising: performing aninverse-transform to the correction signal by additionally using thescaling diagonal matrix.
 10. The method of claim 7, wherein the multiplediagonal matrices includes a diagonal matrix being used to differentiatea weighting of errors in a spatial domain.
 11. The method of claim 7,wherein the correction signal includes an optimal coefficient valuewhich minimizes a sum of a distortion component and a rate component.12. The method of claim 11, wherein: the distortion component isindicative of total distortion between the original video signal and areconstructed signal, and the rate component is indicative of a numberof bits required to send a quantized coefficient.
 13. The method ofclaim 7, wherein the multiple diagonal matrices is read before decodingframes of the video signal.
 14. An apparatus of encoding a video signal,comprising: a receiving unit configured to receive an original videosignal; an optimizer configured to compare the original video signalwith a previously reconstructed signal, and generate a correction signalto minimize a sum of a distortion component and a rate component; and anentropy-encoding unit configured to entropy-encode the correction signalthat is transmitted to a decoder for video signal reconstruction,wherein the previously reconstructed signal has been inverse-transformedby additionally using a scaling diagonal matrix.
 15. The apparatus ofclaim 14, wherein the correction signal is generated based on anotherdiagonal matrix being used to differentiate a weighting of errors in aspatial domain.
 16. The apparatus of claim 14, further comprising: theoptimizer configured to calculate an optimal set of multiple diagonalmatrices including the scaling diagonal matrix, wherein the correctionsignal is generated based on the optimal set of multiple diagonalmatrices.
 17. The apparatus of claim 16, wherein the optimal set ofmultiple diagonal matrices is encoded as side information, and istransmitted to a decoder.
 18. An apparatus of decoding a video signal,comprising: a receiving unit configured to receive the video signalincluding a correction signal, and read side information includingmultiple diagonal matrices from the video signal; an entropy-decodingunit configured to obtain the correction signal by entropy-decoding thevideo signal; and a reconstruction unit configured to reconstruct asignal based on the correction signal and the multiple diagonalmatrices.
 19. The apparatus of claim 18, further comprising: aninverse-transform unit configured to perform an inverse-transform to thecorrection signal by additionally using a scaling diagonal matrix. 20.The apparatus of claim 18, wherein the multiple diagonal matricesincludes a diagonal matrix being used to differentiate a weighting oferrors in a spatial domain.